Goto

Collaborating Authors

 ai system test brilliantly


Grading on a Curve? Why AI Systems Test Brilliantly but Stumble in Real Life

AITopics Custom Links

At a Future of Work forum, experts say demographic shifts, not artificial intelligence, create the biggest challenges...


Grading on a curve? Why AI systems test brilliantly but stumble in real life - ScienceBlog.com

#artificialintelligence

The headline in early 2018 was a shocker: "Robots are better at reading than humans." Two artificial intelligence systems, one from Microsoft and the other from Alibaba, had scored slightly higher than humans on Stanford's widely used test of reading comprehension. The test scores were real, but the conclusion was wrong. As Robin Jia and Percy Liang of Stanford showed a few months later, the "robots" were only better than humans at taking that specific test. Because they had trained themselves on readings that were similar to those on the test.